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ABSTRACT 

Sparse signal recovery has been dominated by the basis pur- 
suit denoise (BPDN) problem formulation for over a decade. 
In this paper, we propose an algorithm that outperforms 
BPDN in finding sparse solutions to underdetermined linear 
systems of equations at no additional computational cost. Our 
algorithm, called WSPGL1, is a modification of the spectral 
projected gradient for £\ minimization (SPGL1) algorithm in 
which the sequence of LASSO subproblems are replaced by 
a sequence of weighted LASSO subproblems with constant 
weights applied to a support estimate. The support estimate 
is derived from the data and is updated at every iteration. The 
algorithm also modifies the Pareto curve at every iteration 
to reflect the new weighted l\ minimization problem that is 
being solved. We demonstrate through extensive simulations 
that the sparse recovery performance of our algorithm is su- 
perior to that of i\ minimization and approaches the recovery 
performance of iterative re-weighted £\ (IRWL1) minimiza- 
tion of Candes, Wakin, and Boyd, although it does not match 
it in general. Moreover, our algorithm has the computational 
cost of a single BPDN problem. 

Index Terms — Sparse recovery, compressed sensing, it- 
erative algorithms, weighted £\ minimization, partial support 
recovery 

1. INTRODUCTION 

The problem of recovering a sparse signal from an underde- 
termined system of linear equations is prevalent in many engi- 
neering applications. In fact, this problem has given rise to the 
field of compressed sensing which presents a new paradigm 
for acquiring signals that admit sparse or nearly sparse repre- 
sentations using fewer linear measurements than their ambi- 
ent dimension JT] |2) . 

Consider an arbitrary signal x £ WL N and let y E 1" be 
a set of measurements given by y = Ax + e, where A is a 
known n x N measurement matrix, and e denotes additive 
noise that satisfies ||e||2 < e for some known e > 0. Com- 
pressed sensing theory states that it is possible to recover x 
from y (given A) even when n <C N, that is, using very few 
measurements. When x is strictly sparse — i.e., when there 
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are only k < n nonzero entries in x — and when e = 0, one 
may recover an estimate x of the signal x by solving the con- 
strained £q minimization problem 



minimize 1 1 it I In, subject to Au 



(1) 



However, £q minimization is a combinatorial problem and 
quickly becomes intractable as the dimensions increase. In- 
stead, the convex relaxation given by the i\ minimization 
problem 



minimize ||tt||i subject to II Au 

uGR N 



yh<e 



(BPDN) 



also known as basis pursuit denoise (BPDN) [3], can be used 
to recover an estimate x. Candes, Romberg and Tao |2] and 
Donoho 1 1 1 show that it is possible to recover a stable and 
robust approximation of x by solving (BPDN) instead of ([1} 
at the cost of increasing the number of measurements taken. 

Several works in the literature have proposed alternate al- 
gorithms that attempt to bridge the gap between £q and £\ 
minimization. These include using £ p minimization with < 
p < 1 which has been shown to be stable and robust under 
weaker conditions than those of £i minimization, see 10] [5] 
|6). Weighted £\ minimization is another alternative if there 
is prior information regarding the support of the signal to-be- 
recovered as it incorporates such information into the recov- 
ery by weighted basis pursuit denoise (w-BPDN) 

minimize ||u||i, w subject to \\Au — y\\2 < e, (w-BPDN) 

u 

where w g (0, 1]^ and ||u||i, w := Yli w i\ u i\ i s tne weighted 

l x norm (see EE] ED). 

When no prior information is available, the iterative 
reweighted £\ minimization (IRWL1) algorithm, proposed by 
Candes, Wakin, and Boyd [10] and studied by Needell ifTTI . 
solves a sequence of weighted £i minimization problems with 



the weights w 



(*) 



V 



„(*"!) 



, where x 



(t-i) 



is the solution 



of the (t - l)th iteration and wf = 1 for all * € {1 . . . N}. 
More recently, Mansour and Yilmaz lfl2l proposed a sup- 
port driven iterative reweighted £\ minimization (SDRL1) 
algorithm that also solves a sequence of weighted £i mini- 
mization problems with constant weig hts wf ] = w e [0, 1] 
when i belongs to support estimates A^) that are updated 
in every iteration. The performance of SDRL1 is shown to 
match that of IRWL1. 



Motivated by the performance of constant weighting in 
the SDRL1 algorithm, we present in this paper an iterative 
algorithm called WSPGL1 that converges to the solution of a 
weighted lx problem (wBPDN) with a two set weight vector 
wa = u) and wa= = 1, where uj G [0, 1] and A is a support 
estimate. The set A to which the algorithm converges is not 
known a priori but is derived and updated at every iteration. 
Our algorithm is a modification of the spectral projected 
gradient for l\ minimization (SPGL1) algorithm [13] which 
solves a sequence of LASSO [14] subproblems to arrive at 
the solution of the BPDN problem. We give an overview of 
the SPGL1 algorithm in section [2] In contrast, our algorithm 
solves a sequence of weighted LASSO subproblems that con- 
verge to the solution of the wBPDN problem with weights 
uj applied to a support estimate A. We discuss the details of 
this algorithm in section [3] and present preliminary recovery 
results in section |4] demonstrating its superior performance 
in recovering sparse signals from incomplete measurements 
compared with l\ minimization. We limit the scope of this 
paper to discussing the algorithm and presenting sparse re- 
covery results and leave the analysis of the algorithm for 
future work. 

Notation: For a vector x £ K. , an index set A C 
{1 . . . N} and its complement A c , let Xk and x\k refer to the 
largest fc entries of x, x(k) is the kth largest entry of x, x\ 
refers to the entries of x indexed by A, and x^ is the vector 
x at iteration t. 

2. THE SPGL1 ALGORITHM 

In this section, we give an overview of the SPGL1 algorithm, 
developed by van den Berg and Friedlander [13], that finds 
the solution to the BPDN problem. 

2.1. General overview 

The SPGL1 algorithm finds the solution of the BPDN prob- 
lem by efficiently solving a sequence of LASSO subproblems 



minimize 

u£R N 



\Au — y\\2 subject to 



< T 



(LSt) 



using a spectral projected-gradient algorithm. The single pa- 
rameter r determines a Pareto curve </>(r) = ||r T ||2, where 
r T = y — Ax T and x T is the solution of (LS T ). The Pareto 
curve traces the optimal trade-off between the least-squares 
fit and the one-norm of the solution. 

The SPGL1 algorithm is initialized at a point x^ which 
gives an initial r = || || i . The parameter r is then updated 
according to the following rule 



Tt+i = T t 



2-e 



IIA^IU/Hmii 



(2) 



where superscript H indicates Hermitian transpose, and e = 
IMh = \\y — Ax\\2- Consequently, the next iterate x^ t+1 ^ is 




Fig. 1: Example of a typical Pareto curve showing the root 
finding iterations used in SPGL1 ff3l . 



given by the solution of (LS Tt+1 ) and the algorithm proceeds 
until convergence. 

2.2. Probing the Pareto curve 

One of the main contributions of lTT~3l lies in recognizing and 
proving that the Pareto curve is convex and continuously dif- 
ferentiable over all solutions of (LS r ). This gives rise to the 
update rule for r shown in |2} and guarantees the convergence 
of SPGL1 to the solution of BPDN. 

The update rule (|2]l is in fact a Newton-based root-finding 
method that solves 0(t) = e. The update rule generates a 
sequence of parameters r t according to the Newton iteration 



T~t+1 = T t 



where 0'(r 4 ) is the derivative of <fi at Tt. It is then shown that 
the </>'(t) is equal to the negative of the dual variable A of 

(LS T ) resulting in the expression 4>'{t) = —A = — ^"1i r n ■ 



Figure[T]illustrates an example of a Pareto curve and the root 
finding method used in SPGL1. 

3. THE PROPOSED WSPGL1 ALGORITHM 

In this section, we describe the proposed WSPGL1 algorithm 
for sparse signal recovery as a variation of the SPGL1 algo- 
rithm. The WSPGL1 algorithm solves a sequence of weighted 
LASSO subproblems to arrive at the solution to a weighted 
BPDN problem with weights u 6 [0, 1] applied to a support 
set A. The set A is derived and updated from the solutions of 
the weighted LASSO subproblems (LS T , w). 

3.1. Algorithm description 

The two algorithms SPGL1 and WSPGL1 follow exactly the 
same initial steps until the solution x Tl of the first LASSO 



problem (LS Tl ) is found. At this point, WSPGL1 generates a 
support set A containing the support of the k largest in magni- 
tude entries of x Tl . A weight vector w is then generated such 
that 

f w, i e A 
w * = \ 1, i e A c 

We heuristically choose k = n/ (2 log(A/n)) and cj = 0.3. 

The weight vector is then used to define the weighted 
LASSO subproblem 

minimize II Au — y\U subject to ||u||i w < r (LS rw ) 
with the corresponding dual variable 

w — n n ' 

rib 

where ||w||oo,w = \\ v • w_1 ||oo- The weighted LASSO sub- 
problem and its dual constitute a subproblem of (wBPDN) 
with support estimate A. The BPDN and wBPDN prob- 
lems have different Pareto curves. Therefore, the iterate 
(II ' II 2, t~x) which lies on the Pareto curve of BPDN must be 
adjusted to lie on the Pareto curve of the wBPDN prob- 
lem. This can be easily achieved by switching t\ with 
t{ = ||:r Tl || i jW . The WSPGL1 algorithm then proceeds 
according to the following pseudocode. 



Algorithm 1 The WSPGL1 algorithm 

1: Input y = Ax + e, e, k = n/ (2 log(JV/n)), u G [0, 1] 

2: Output xW 

3: Initialize wf ) = 1 for all i € {1 ... N} 

t = 0, x<® = 0, r = 

4: loop 

5: t = t + l 

6: A = SUpp(x^\ k ), W, = | . l |^ c 
8: T t = r t-l + || J 4»r T t-i|| 0O , w /||r T t-i|| 2 

9: x^*) = argmin \\Au — j/|| 2 s.t. ||u||i. w < r t 

10: r Tt = y - Ax^ 
11: end loop 



3.2. Discussion 

The WSPGL1 algorithm converges to the solution of a 
weighted BPDN problem with weights uj <G [0, 1] applied 
to a support set A. When the sparse signal is recovered ex- 
actly, the set A coincides with the true support of the sparse 
signal x. Figure[2](a) illustrates the solution path of WSPGL1 
which follows the Pareto curve of the BPDN problem until 
the first (LS T ) is solved. The algorithm then uses the sup- 
port information from x Tl to switch to the Pareto curve of 
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Fig. 2: (a) The solution path for WSPGL1 follows the 
BPDN Pareto curve until the first (LS T ) is solved, after which 
WSPGL1 switches to the wBPDN Pareto curve, (b) Solution 
paths of WSPGL1, SPGL1, and weighted SPGL1 with oracle 
support information. Both WSPGL1 and the oracle weighted 
SPGL1 use lu = 0.3. 

the wBPDN problem. Figure [2] (b) compares the solution 
paths of WSPGL1, SPGL1, and oracle weighted SPGL1 with 
weight uj = 0.3 applied to the true signal support. It can be 
seen that WSPGL1 converges to the solution of the oracle 
weighted l\ problem. Moreover, the solution paths of these 
algorithms merge after only the first (LS r ) subproblem. Note 
here that the x-axis is the parameter r which is equal to the 
one-norm of x^' for SPGL1 and the weighted one-norm of 
for WSPGL1 and the oracle weighted SPGL1. 

It is still not clear under what conditions the WSPGL1 
algorithm achieves exact recovery. What is clear is that 
WSPGL1 can exactly recover signals with far more nonzero 
coefficients than what BPDN can recover. The WSPGL1 
algorithm is motivated by the work in [9] and 1121 . which 
show that weighted £i minimization can recover less sparse 
signals than BPDN when the weights are applied to a support 




Fig. 3: Comparison of the percentage of exact recovery of sparse signals between the proposed WSPGL1, SDRL1 [ 12], IRL1 
ifTUl . and standard l\ minimization using SPGL1 1 13 1. The signals have an ambient dimension N — 2000 and the sparsity and 
number of measurements are varied. The results are averaged over 100 experiments. 



estimate that is at least 50% accurate. Moreover, it is pos- 
sible to draw a support estimate from the solution of BPDN 
and improve that support estimate by solving wBPDN us- 
ing the initial support estimate. Based on these results, we 
conjectured that the solution of every LASSO subproblem in 
SPGL1 allows us to find a support estimate that is accurate 
enough to improve the recovery conditions of the correspond- 
ing wBPDN problem. A full analysis of this algorithm will 
be the subject of future work. 

4. NUMERICAL RESULTS 

We tested the WSPGL1 algorithm by comparing its perfor- 
mance with SDRL1 1 12 1, IRWL1 GO) and standard l x mini- 
mization using the SPGL1 [13] algorithm in recovering syn- 
thetic signals x of dimension A = 2000. We first recover 
sparse signals from compressed measurements y — Ax using 
matrices A with i.i.d. Gaussian random entries and dimen- 
sions n x A where n € {A/10, A/4, A/2}. The sparsity of 
the signal is varied such that k/n € {0.1, 0.2, 0.3, 0.4, 0.5}. 
To quantify the reconstruction performance, we plot in Figure 
[3]the percentage of successful recovery averaged over 100 re- 
alizations of the same experimental conditions. The figure 
shows that in all cases, the WSPGL1 algorithm outperforms 
standard ix minimization in recovering sparse signals. More- 
over, the recovery performance approaches that of the itera- 
tive reweighted i\ algorithms SDRL1 and IRWL1 while re- 
quiring only a fraction of the computational cost associated 
with these algorithms. 
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